E. Rantsiou, 15/02/2021, LENS ML school 2021

Table of Contents

• 1  Convolutional Neural Networks (CNN, ConvNet)
  • 1.1  Why CNNs?
  • 1.2  Convolution Operation
  • 1.3  Example with edges filters
  • 1.4  Padding
  • 1.5  Strided Convolutions
  • 1.6  Convolutions on volumes (RGB images)
  • 1.7  Layers in ConvNets
  • 1.8  Schematic Description of a ConvNet
  • 1.9  CNNs in Tensorflow
• 2  Hands-on
  • 2.1  steps
  • 2.2  Dropout
  • 2.3  Augmentation
  • 2.4  Transfer Learning

import numpy as np 
import scipy
from scipy import signal
import pandas as pd 
import os
import glob
import matplotlib
#matplotlib.use('Agg')
import matplotlib.pyplot as plt
from time import time
import PIL.Image
from numpy import asarray
from IPython.display import Image,IFrame

Convolutional Neural Networks basics

and an example with classification

(LENS ML School 2021; Emmanouela Rantsiou)

Convolutional Neural Networks (CNN, ConvNet)

• a class of artificial Neural Networks
• widely used in computer vision*
• learn hierarchy of features

*Computer vision is an essential, complex, wide-spread, and ever developing part of AI. Computer vision tasks include:

• Image Classification
• Classification with localization
• Object detection
• Image captioning
• action classification
• Semantic segmentation
• Instance segmentation
• Neural Style Transfer

Why CNNs?

• Need not to be restricted to small input images.

• Image of 1300 X 1300px --> Input on a layer of a FC (Dense) network with e.g. 1000 hidden units $\rightarrow$ parameters (weights) to calculate : (1000,1300x1300) = 1.69E9 parameters (x3 for a color image) $\rightarrow$ Computationally expensive and data hungry.

Convolution Operation

Example with edges filters

(edge detectors)

filter_v=np.array([[1,0,-1],[1,0,-1],[1,0,-1]])  # 3x3 vertical edges filter 
filter_h=np.array([[1,1,1],[0,0,0],[-1,-1,-1]])  # 3x3 horizontal edges filter 
filter_v_3d=np.dstack([filter_v]*3)                 # 3x3x3 vertical edges filter 
filter_h_3d=np.dstack([filter_h]*3)                 # 3x3x3 horizontal edges filter 

image = PIL.Image.open('brick2.jpeg').convert('L')
brick = asarray(image)
plt.figure(figsize=(20,20))
plt.imshow(brick,cmap='gray')
plt.axis('off')
plt.show()

image = plt.imread('filters_conv.jpeg')
plt.figure(figsize = (20,20))
plt.imshow(image)
plt.axis('off')
plt.show()

out_conv_h=signal.convolve2d(brick,filter_h,mode='valid')
out_conv_v=signal.convolve2d(brick,filter_v,mode='valid')
f, axarr = plt.subplots(1,2,figsize = (50,50))
axarr[0].imshow(np.absolute(out_conv_h),cmap='gray')
axarr[0].set_axis_off()
axarr[1].imshow(np.absolute(out_conv_v),cmap='gray')
axarr[1].set_axis_off()

plt.show()

out_conv_v=signal.convolve2d(brick,filter_v,mode='valid')
out_conv_h=signal.convolve2d(brick,filter_h,mode='valid')

f, axarr = plt.subplots(1,2,figsize = (50,50))
axarr[0].imshow(np.absolute(out_conv_h),cmap='gray')
axarr[1].imshow(np.absolute(out_conv_v),cmap='gray')
plt.show()

image = PIL.Image.open('brick.jpeg')
brick = asarray(image)
plt.figure(figsize=(10,10))
plt.imshow(brick)
plt.axis('off')
plt.show()

out_corr_v=signal.correlate(brick,filter_v_3d,mode='valid')
out_corr_h=signal.correlate(brick,filter_h_3d,mode='valid')

f, axarr = plt.subplots(1,2,figsize = (50,50))
axarr[0].imshow(np.absolute(out_corr_h),cmap='seismic')
axarr[0].set_axis_off()
axarr[1].imshow(np.absolute(out_corr_v),cmap='seismic')
axarr[1].set_axis_off()
plt.show()

Padding

Strided Convolutions

Convolutions on volumes (RGB images)

Layers in ConvNets

The building blocks (i.e. most commonly used types of layers in CNNs)

• Convolutional layers
• Pooling layers
  • Max Pooling
  • Average Pooling
  • Global Pooling
• Dense (Fully Connected) layers

Schematic Description of a ConvNet

CNNs in Tensorflow

import tensorflow
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras import models

model = models.Sequential()
model.add(layers.Conv2D(8, (3, 3), activation='relu', input_shape=(80,80,3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(16, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(64,activation='relu'))
model.add(layers.Dense(4, activation='softmax'))

Hands-on

• Simple classification example using a ConvNet

Classification/Regression

• Regression:
  • output is a prediction on a quantity which takes continuous values
• Classification:
  • output is a prediction on a class, i.e. discrete labels
    • binary classification
    • multi-class classification
      • single-label classification
      • multi-label classification

steps

1. Data
   • set of 8 different classes of images: airplane, car, cat, dog, flower, fruit, motorbike, person
   • create the relevant datasets
2. CNN model
   • build the CNN model, train it with the given data, check performance
   • we will observe overfitting
3. Improve previous CNN model: i.e. avoid overfitting
   • Dropout (interfere in the model architecture)
   • Augmentation (interfere with the data)
4. Transfer learning
   • Do not use own CNN model, but borrow a pretrained one. Use it on our given data.

Dropout

Dropout: A Simple Way to Prevent Neural Networks from Overfitting, JMLR (2014) http://jmlr.org/papers/v15/srivastava14a.html

Augmentation

Augmenting the training dataset, by applying transformations to existing data (images in this case): flip, rotate, shear, shift, zoom, color distortion, etc.

Transfer Learning